This invention relates to a non-intrusive video quality assessment system.
Signals carried over telecommunications links can undergo considerable transformations, such as digitisation, encryption and modulation. They can also be distorted due to the effects of lossy compression and transmission errors.
Objective processes for the purpose of measuring the quality of a signal are currently under development and are of application in equipment development, equipment testing, and evaluation of system performance.
Some automated systems require a known (reference) signal to be played through a distorting system (the communications network or other system under test) to derive a degraded signal, which is compared with an undistorted version of the reference signal. Such systems are known as “intrusive” quality assessment systems, because whilst the test is carried out the channel under test cannot, in general, carry live traffic.
Conversely, non-intrusive quality assessment systems are systems which can be used whilst live traffic is carried by the channel, without the need for test signals.
Non-intrusive testing is required because for some testing it is not possible to use test signals. This could be because the termination points are geographically diverse or unknown. It could also be that the cost of capacity is particularly high on the route under test. Whereas, a non-intrusive monitoring application can run all the time on the live data transmissions to give a meaningful measurement of performance.
A known non-intrusive quality assessment system uses a database of distorted samples which has been assessed by panels of human listeners to provide a Mean Opinion Score (MOS).
MOSs are generated by subjective tests which aim to find the average user's perception of a system's quality by asking a panel a directed question and providing a limited response choice. For example, to determine quality of transmitted coded video users may be asked to rate “the quality of the video” on a five-point scale from Bad to Excellent. The MOS, is calculated for a particular condition by averaging the ratings of all panel members.
In order to train a quality assessment system a sample is parameterised and a combination of the parameters is determined which provides the best prediction of the MOSs indicted by the panel.
Known methods of video quality assessment such as that described in S. Murphy, C. Rambeau, M. Searles, L. Murphy, “Evaluating the Impact of Network Performance on Video Streaming Quality for Categorised Video Content,” presented at Packet Video 2004 (PV2004), Irvine, Calif., USA, 13-14 Dec. 2004 consider the effect of packet loss and jitter, or consider the effects of frame rate of bit rate. However, such measures are only able to take into account degradation due to network conditions. It is desirable to use parameters which are able to measure the degradation effect due to the video compression itself. In particular it is desirable to use a parameter which takes into account the temporal and/or the spatial characteristics of a stream of video frames in which each video frame is encoded such that it is represented by one or more packets.
In SLUNUELI et al “Effects of frame rate, frame size and MPEG2 compression on the perceived compressed video quality transmitted over lossy tp networks” ISBN 0-7803-8625-6 the relationship between viewer perceived quality and encoding schemes is investigated. The encoding schemes depend upon MPEG compression depth, frame size and frame rate. In this paper the frame size and frame rate are considered prior to coding the data in order to determine an optimal coding scheme.
Both EP 1 530 200 and EP 1433 497 present parameters suitable for use in a quality assessment tool in which parameters are formed over a number of packets ie over a period of time.